ci: trace the linux-aarch64 symbol order file on every main build - #37711

### Problem
- `darwin-aarch64` and `linux-x64` builds link with a traced symbol
order file; Windows release builds link without one, so `windows-x64`
and `windows-aarch64` miss the startup win.
- The existing tracer is an `LD_PRELOAD` library, which has no Windows
counterpart, and a Windows executable has no symbol table to map traced
addresses back to names.
- Related: #37711, #35328.

### Fix
- Windows release links take `/order:@linker.order` (plus `/ignore:4037`
for traced names that no longer exist), and both windows targets get a
`trace-order` step on their existing test images. The generated pipeline
differs by exactly those two steps.
- A new tracer runs the binary as its debuggee: with the process stopped
at creation it writes a breakpoint over every function start; each hit
restores the instruction, records the address and resumes there. Every
function traps once and then runs unmodified, and the trace file format
is the unix tracer's.
- Names come from two maps the release link now writes into the profile
zip: `/map` for names, `/lldmap` for where each input chunk landed. Only
names at chunk starts count as functions, because the MSVC CRT leaves
labels (`Table`, `$LN123`) on data inside `.text` and a breakpoint there
killed the debuggee. Chunk starts are exactly what `/order` can move, so
nothing orderable is lost.
- Verification: the linker-order test covers the flag and map wiring,
both map parsers against fixtures, and on windows the tracer itself
against a C fixture (exact entry order, exit code, argv, ConPTY stdio).
The generator was also run by hand on Server 2019 x64 and Windows 11
arm64 boxes, all eight workloads under the real tracer, and this PR's
own build runs all four trace-order steps on the real binaries.

### Background
- Symbol order file: one function name per line; the linker lays those
out first in `.text` so startup-hot code shares pages. Generated in CI,
never committed. lld takes `--symbol-ordering-file`, Apple ld
`-order_file`, lld-link `/order:@file`.
- trace-order step: a soft-fail, non-PR CI step that downloads the build
lane's unstripped `bun-profile`, runs a few workloads under a tracer
that records first-entry order, and uploads the `.order` file the next
build links with.
- `/order` can only move a function that is in its own COMDAT section.
Bun's C++ and the WebKit prebuilt use `/Gy`, rustc emits function
sections, and LTO output is per-function, so all of bun's code already
qualifies.
- ELF and Mach-O binaries carry a symbol table that `nm` reads. A PE
keeps its names in the PDB, so on Windows the linker's map files stand
in as the address-to-name source.
- A Windows debugger gets its debuggee's creation event before any of it
has run, may write its memory, and receives every breakpoint exception,
so nothing has to be loaded into the traced process.

<!-- robobun:evidence:begin -->

---

**no test proof** · iteration 0 · Platform-specific test-only change;
deferring to CI.

<!-- robobun:evidence:end -->

<details>
<summary>Original description</summary>

## What

`darwin-aarch64` and `linux-x64` have a `-trace-order` step that traces
the freshly built `bun-profile` on the native test fleet and publishes
the symbol order file the next build links with. Windows had none of
this: `usesOrderFile()` was false there, and the tracer is an
`LD_PRELOAD` library. This adds it for `windows-x64` and
`windows-aarch64`.

- **Link** (`scripts/build/flags.ts`): windows release links get
`/order:@<buildDir>/linker.order` plus `/ignore:4037` (the
`--no-warn-symbol-ordering` counterpart), and `linker.order` becomes a
link dependency so an inherited file relinks. Every function is its own
COMDAT already (`/Gy` for bun's C++ and the WebKit prebuilt, function
sections for rustc, per-function LTO output), which is what `/order`
needs; checked against lld-link 21 that it orders statics and
ThinLTO-internalized functions too.
- **Tracer** (`scripts/orderfile/functrace-windows.c`): a debugger
rather than an injected library. It starts the binary as its debuggee,
writes a breakpoint (x64 `INT3`, arm64 `BRK #0xF000`) over every
function start while the process is stopped at its creation event, and
on each breakpoint exception restores the instruction, records the
address and resumes the thread there. Everything else (the loader's
initial breakpoint, exceptions the program handles itself) is passed
through. With `BUN_FUNCTRACE_TTY` it starts the debuggee on a ConPTY and
types our stdin into it, standing in for `ptyrun.c`. Same trace file
format as `functrace.c`. Built on the test VM by `generate.ts` with
clang-cl (on the images' machine PATH) or cl.
- **Symbols** (`scripts/orderfile/windows-symbols.ts`): a PE has no
symbol table, so the windows release link writes two maps that ship in
the profile zip next to `bun-profile.pdb`: lld-link's MSVC-style `/map`
(`bun-profile.map`, every symbol under the exact name `/order` takes)
and `/lldmap` (`bun-profile.linker-map`, where every input chunk was
placed). Only names at chunk starts are treated as functions. This
turned out to be necessary, not cosmetic: the MSVC CRT defines symbols
on data it keeps in `.text`. The arm64 `memset` keeps the byte table for
its computed branch there under a symbol named `Table`, and
MSVC-compiled code leaves a `$LN123` symbol on every slot of the jump
tables it emits after a function. A breakpoint written over those
corrupts the table and the debuggee died in the first small `memset` /
in `printf`. Chunk starts are also exactly the set `/order` can move, so
the filter loses nothing orderable. Cost: roughly 10 to 15 MB more per
compressed windows profile zip (the linux map compresses about 10x).
- **CI** (`.buildkite/ci.mjs`): `windows-x64` and `windows-aarch64` join
`traceOrderTargets`, tracing on the existing Server 2019 x64 and Windows
11 arm64 test images. The step runs under cmd.exe through
`vs-shell.ps1`, like the test runner. Generating the pipeline as main
before and after this change differs by exactly those two steps.
- `linkerMapPath` moves from `rust.ts` to `flags.ts` next to the new
`symbolMapPath`/`linkerMapOutputs`; `bun.ts` declares the maps to ninja
and `ci.ts` packs them from that one list. `verifyOrderFileApplied()`
now reads symbols through the same `readTextSymbols()` the generator
uses, so it works on the windows build lanes too.

## Verification

`test/js/bun/perf/linker-order.test.ts` (26 pass / 4 platform skips on
each of linux x64, windows x64 and windows arm64):

- flag, link-dependency and map-output wiring for both windows targets,
and that the maps are written exactly where `linkerMapOutputs()`
declares them;
- parsers for both maps against fixtures shaped like real output (long
names, folded functions, absolute symbols, demangled lld-map names,
`Table`/`$LN` labels being dropped);
- the tracer itself, windows only: `functrace-fixture.c` (now also
compiled with clang-cl; it links the static CRT, so it crashes on arm64
without the chunk filter) must record `f0..f31` in order, `main`, and
`after` as distinct first entries with a child process spawned in
between; the debuggee's exit code and argv come through and a starts
file for another binary is refused; and a C probe reports console stdio
80 columns wide and the typed line under `BUN_FUNCTRACE_TTY`, pipe stdio
otherwise. The unix tracer test now makes the same exact-order
assertions.

Also run by hand on the two windows boxes (Server 2019 x64, Windows 11
24H2 arm64, both bootstrapped like the CI images): the whole of
`generate.ts` against the installed `bun.exe` with maps faked down to
its entry point, so all eight workloads run under the real tracer,
ConPTY one included (`cli-fixture.js` reports `tty=true 80x24` and exits
on `quit`); and a 20k-function program to size the cost, about 65 to 70
us per breakpoint hit on both, so a full trace of bun is a few seconds.
The unix path was re-run against a linux release build (13973
functions).

The commit message carries the opt-in tag `ci.mjs` honours outside main,
so this PR's own build runs all four trace-order steps; the two windows
ones are the end-to-end check on the real binaries and maps. The tag is
deliberately not in this title or body.

Related: #37711 (linux-aarch64 step), #35328.

</details>

)

### Problem
- Some builds of main ship unordered binaries.
[#113190](https://buildkite.com/bun/bun/builds/113190) and
[#112691](https://buildkite.com/bun/bun/builds/112691) linked linux-x64,
darwin-aarch64 and both windows targets with an empty order file (`~
symbol order: found no earlier build to inherit from (34s)`). Unordered,
`bun hello.js` keeps about 22 MB of text resident, not 7 MB.
- The cause is `candidateBuilds()` (`scripts/build/ci.ts:827`). It
appends `.json` to the `Location` of Buildkite's
`builds/latest?branch=main&state=passed` redirect. Since 2026-08-21 the
`Location` repeats the query, so the request is
`<build>?branch=main&state=passed.json`. Buildkite answers with HTML,
and the lookup returns no build.
- That leaves the probe of the 200 build numbers below the current one.
PR builds use numbers too, so a quiet night exceeds 200.

### Fix
- Put `.json` on the path of the redirect target.
- Probe first, and use the newest passed build as the fallback. The
nearest file matches the link best. When the newest passed build came
first, one stale file served every build until main passed again:
[#102154](https://buildkite.com/bun/bun/builds/102154) and
[#102263](https://buildkite.com/bun/bun/builds/102263) both resolved 506
of the 1000 hottest names.
- Verified: `test/js/bun/perf/linker-order.test.ts` (six new tests, the
fallback test fails with the old URL). Run for real from #113190, the
walk now yields the newest passed build.

### Background
- A symbol order file lists the functions that run at start-up. The
linker places them together, so a start touches fewer pages.
- Most release lanes cross-compile and cannot trace their own binary. A
`trace-order` step traces it on a native host and publishes
`bun-<target>.order`. The next build of main inherits that file.
- Buildkite has no public list of a branch's builds, so
`candidateBuilds()` probes build numbers.

<details><summary>Notes</summary>

**How often.** I read the `Inherit symbol order file` group of the
linux-x64 build step for 35 builds of main from 09-05 to 09-16. 33
inherited a file. #112691 (`none of the 3 builds tried published it
(33s)`) and #113190 did not, on any lane. In both, linux-aarch64, which
builds natively, paid the second link (`Tracing bun-profile to build a
fresh order file (nothing to inherit)`). #112691 is a passed build, so
its binaries were uploaded as the canary. A sample of August builds has
a third case: [#103671](https://buildkite.com/bun/bun/builds/103671) on
08-22 (`found no earlier build to inherit from (37s)`).

**When it broke.** The lookup worked from #33345 (07-05) until 08-21.
#69595 inherited from #68645, 950 build numbers back, in 1s.
[#102263](https://buildkite.com/bun/bun/builds/102263) (08-21 04:33Z)
inherited from #102067 in 0s.
[#102418](https://buildkite.com/bun/bun/builds/102418) (08-21 06:54Z)
took 33s to inherit from #102263, 155 numbers back. No commit touched
`scripts/build/ci.ts` or `scripts/utils.mjs` in that window, so the
redirect changed on Buildkite's side.

**What it looks like since.** The time to inherit follows the distance
to the build inherited from, at about 0.16 s per build number on top of
a fixed 6 s. The fixed part is `utils.curl` retrying the JSON parse of
the HTML page three times, with sleeps of 2 s and 3 s.

| build | inherited from | distance | time |
| --- | --- | --- | --- |
| #115612 | #115611 | 1 | 6s |
| #116354 | #116338 | 16 | 9s |
| #114270 | #114150 | 120 | 23s |
| #115994 | #115833 | 161 | 31s |
| #111080 | #110907 | 173 | 35s |

```
$ curl -sI 'https://buildkite.com/bun/bun/builds/latest?branch=main&state=passed' | grep -i ^location
location: https://buildkite.com/bun/bun/builds/116199?branch=main&state=passed
$ curl -s -o /dev/null -w '%{content_type}\n' 'https://buildkite.com/bun/bun/builds/116199?branch=main&state=passed.json'
text/html; charset=utf-8
$ curl -s -o /dev/null -w '%{content_type}\n' 'https://buildkite.com/bun/bun/builds/116199.json'
application/json; charset=utf-8
```

**Why the probe goes first.** The code on main asks the newest passed
build first. With only the URL corrected, every build of main inherits
from the newest passed build again, and not from the nearest build. Most
builds of main fail a test or get cancelled, so that file is often many
commits old. Rust symbol names (v0 mangling) embed a hash per crate. The
hash changes when the dependencies of the crate change, or when the
toolchain changes, and then the names in an older file no longer
resolve. The `+ symbol order: applied` line of the linux-x64 build step
shows both orders:
- Newest passed first (until 08-21). #100959, #100992, #101024 and
#101032 all inherited the file of #100916 and resolved 847, 847, 843 and
843 of 1000. #102154 and #102263 both inherited the file of #102067 and
resolved 506 of 1000.
- Nearest first (since 08-21, by accident). Of the 33 sampled builds
that inherited, 31 resolved 947 or more. #112799 resolved 515 (its
parent commit, 8b672e4, removes 21 dependency edges), and the next
build, #112830, resolved 990. #116425, the first build after the LLVM 23
and Rust nightly upgrade, resolved 443.

The probe costs about 30 s when the window is empty, in a build step
that takes many minutes.

**The fallback is one candidate.** The `trace-order` step is
`soft_fail`, so a passed build can lack the file. In that case nothing
is inherited, as today. The cap of 50 candidates now counts inside the
probe, so a crowded window cannot keep the newest passed build from
being asked.

**Releases.** A release build of a cross-compiled lane inherits the same
way. The linux-x64 build of v1.4.2 (#110323) inherited from #110317. A
release cut more than 200 build numbers after the previous build of main
ships unordered on those lanes.

**The seam.** `BuildLookups` follows `OrderFileContext` in the same
file: the facts come in as an argument so that the decision runs in a
test. The default value makes the same two requests as before.
`inheritOrderFile()` is the only caller and uses the default.

**Related.** #37711 touches the same files for a different problem (a
`trace-order` step for linux-aarch64). With it, a lane that inherits
nothing links unordered and no longer traces itself, so this fallback
matters more.

**Not in this PR.** `curl()` in `scripts/utils.mjs` keeps the `error` of
a failed attempt after a retry succeeds, so a build whose JSON needed
one retry is skipped here. The bug is from 2024 and affects every caller
of `curl()` and `curlSafe()`. It gets its own PR.

**Self-review.** A review of this description raised three points. One
was a wrong claim in my draft, that the lookup never worked: the July
and August logs show that it worked until 08-21, and the text above says
so. The other two are the single fallback candidate and the cost of the
probe. I accept both, for the reasons above.

**Test runs.** `bun bd test test/js/bun/perf/linker-order.test.ts`: 32
pass, 4 skip (windows only), 0 fail. I ran the real walk against
Buildkite under bun and under Node 26, which is what CI runs the build
scripts with. From #116354 it yields #116353, #116352 and #116338 within
3 s, and CI inherited from #116338. From #113190 it finds no build of
main in the window and then yields the newest passed build. `bunx tsc
--noEmit -p scripts/build/tsconfig.json` reports the same four errors as
main, all in `jsonByteClass.ts` and `xmlByteClass.ts`.

</details>

<!-- robobun:evidence:begin -->

---

**no test proof** · iteration 0 · platform-specific test(s) that do not
run on this machine, deferring to CI, which covers all platforms:
test/js/bun/perf/linker-order.test.ts

<!-- robobun:evidence:end -->